Understanding the Limitations of Estimation Methods for Long-Range Dependence

Over the last ten years, long-range dependence (LRD) has become a key concept in modeling networking phenomena. The research community has undergone a mental shift from Poisson and memoryless processes to LRD and bursty processes. Despite its popularity, LRD analysis is hindered by two main problems: a) it cannot be used by nonexperts easily, and b) the identification of LRD is often questioned and disputed. The main cause for both these problems is the absence of a systematic and unambiguous way to identify the existence of LRD. This paper has two main thrusts. First, we explore the (lack of) accuracy and robustness in LRD estimation. We find that the current estimation methods can often be inaccurate and unreliable, reporting LRD erroneously. We search for the source of such problems and identify a number of caveats and common mistakes. For example, some of the methods misinterpret short-range correlations for LRD. Second, we develop methods to improve the robustness of the estimation. Through case studies, we demonstrate the effectiveness of our methods in overcoming most known caveats. Finally, we integrate all required functionality and methods in an easy to use software tool. Our work is a first step towards a systematic approach and a comprehensive tool for the reliable estimation of LRD.